Skip to content

gpu-config [2/13]: identify driver loads and driver readiness - #3248

Open
henry118 wants to merge 2 commits into
NVIDIA:mainfrom
henry118:gpu-config/02-loadid
Open

henry118 wants to merge 2 commits into
NVIDIA:mainfrom
henry118:gpu-config/02-loadid

Conversation

@henry118

@henry118 henry118 commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Part of #3246 (2/13). Depends on: none. Review only the last commit.

Adds internal/gpuconfig/loadid: the driver load ID (the inode of
/sys/module/nvidia) and the driver readiness check, based on the
validator's ready files. Not used yet.


Devin Review

@henry118
henry118 requested a review from a team as a code owner November 17, 2026 18:28

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@coderabbitai

coderabbitai Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Explain Change →

Walkthrough

The change adds a loadid package that reads the NVIDIA module inode and checks driver metadata and readiness. It also adds a statefile package with a versioned apply and gate state model, state predicates, and locked reads and writes that replace the state file atomically.

Merge Risk: Low · up to dcce9

This change adds helper packages that are not yet wired into anything, so there is no current user impact. A malformed state file with duplicate version keys could later leave the state file unreadable. Two tests would not catch regressions in read-error handling and write-base invalidation. These are small, low-risk follow-ups.

Security architecture risk: Minimal · up to dcce9

The new contracts warrant design review, but the helpers are not yet connected to production actions. The inspected change establishes no material security regression or weakened failure-containment guarantee.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The intended state scope is node-local GPU configuration and allocator gating, not tenant or cluster-wide persistence. Production consequences remain unwired in this revision, so the exported Go APIs do not by themselves establish an externally attackable entrypoint.

Trust Boundaries and Controls

  • observed — Ownership detection trusts label-selected DaemonSets in a caller-selected namespace and their controller API group and kind. It does not verify the owner's existence or UID. Readiness also trusts local marker-file contents. These are metadata and filesystem trust assumptions, not authentication controls; no new privileged consumer was found.

Hardening Proposals

  • proposed — When integrating privileged callers, explicitly preserve module-identity freshness across readiness checks, GPU actions, and decision writes. Establish directory ownership and recovery behavior as part of that integration rather than treating marker files or advisory locking as authentication.
- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.

Other comments: 1

internal/gpuconfig/statefile/state.go: 1

101-101: Minor · Data Integrity & Integration · Quick win

Reject inconsistent decoded and retained state.

{"version":1,"version":null} passes the State decode because JSON null leaves the integer unchanged. The raw-map decode retains only version:null. These behaviors follow Go 1.26.3 decoding rules. (raw.githubusercontent.com)

After a successful Lock.Read, WriteGatePass(7) renames content containing version:null over the state file. Its final parse then fails. Later reads return ErrParse, and writes remain blocked.

Decode State from the same normalized representation used for writes, or reject ambiguous keys before accepting the file. Preserve ordinary version-1 files and unknown top-level keys.

Nitpick comments: 3

internal/gpuconfig/loadid/loadid_test.go: 2

202-202: Trivial · Maintainability & Code Quality · Quick win

Check the selector against the literal label contract.

Production and this assertion use DriverDaemonSetSelector. If that constant changes to an incorrect selector, every fake-lister case still passes. A real list could then miss operator-managed driver DaemonSets.

Assert the literal value "app.kubernetes.io/component=nvidia-driver" instead.

As per path instructions, “Flag tests that mirror the implementation instead of checking behavior.”

Source: Path instructions


157-157: Trivial · Maintainability & Code Quality · Quick win

Cover the non-ENOENT read-error path in the Ready tests.

TestReadyErrorOnUnreadableDriverReady uses a readable file, so it exercises only the parse-error branch. If readDriverInfo suppressed non-ENOENT read errors, such as permission failures, this test would still pass. The test name also does not match its fixture.

Keep this parser-error case. Add a case where DriverReadyFile is a directory. os.ReadFile then fails with EISDIR, which readDriverInfo must report as an error with ok=false:

Suggested test

func TestReadyErrorOnDriverReadyRead(t *testing.T) {
	dir := t.TempDir()
	require.NoError(t, os.Mkdir(filepath.Join(dir, DriverReadyFile), 0o700))
	info, ok, err := Ready(dir, true)
	require.Error(t, err)
	require.False(t, ok)
	require.Nil(t, info)
}
internal/gpuconfig/statefile/file_test.go: 1

214-214: Trivial · Maintainability & Code Quality · Quick win

Cover a failed read after a successful read in the same hold.

TestWriteAfterAFailedReadFails starts with l.base == nil. A Lock.Read that keeps an earlier successful base after a failed read would leave the write refused, so this test would still pass. The test does not exercise the state that the regression changes.

Rewrite the test so it:

  1. Writes a valid state file and calls l.Read() successfully in the same hold.
  2. Replaces the file with {"version":2}.
  3. Calls l.Read() again and requires ErrParse.
  4. Requires l.WriteGatePass(7) to fail.
  5. Requires the file contents to remain {"version":2}.

With this sequence, a Read that keeps the earlier base can let WriteGatePass overwrite the corrupted file. The final content assertion then fails.

Review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 246a9ca and dcce9b9.

Files selected for processing: 7

  • internal/gpuconfig/loadid/driver.go
  • internal/gpuconfig/loadid/loadid.go
  • internal/gpuconfig/loadid/loadid_test.go
  • internal/gpuconfig/statefile/file.go
  • internal/gpuconfig/statefile/file_test.go
  • internal/gpuconfig/statefile/state.go
  • internal/gpuconfig/statefile/state_test.go

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

Signed-off-by: Henry Wang <henrwang@nvidia.com>
Signed-off-by: Henry Wang <henrwang@nvidia.com>
@henry118
henry118 force-pushed the gpu-config/02-loadid branch from ce3ec01 to a795268 Compare November 19, 2026 00:16
@henry118 henry118 changed the title gpu-config: identify driver loads and driver readiness gpu-config [2/13]: identify driver loads and driver readiness Nov 19, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant